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A scheme to provide various mean-field-type approximation algorithms is presented by 
employing the Bethe free energy formalism to a family of replicated systems in conjunction 
with analytical continuation with respect to the number of replicas. In the scheme, survey 
propagation (SP), which is an efficient algorithm developed recently for analyzing the micro- 
scopic properties of glassy states for a fixed sample of disordered systems, can be reproduced 
by assuming the simplest replica symmetry on stationary points of the replicated Bethe free 
energy. Belief propagation and generalized SP can also be offered in the identical framework 
under assumptions of the highest and broken replica symmetries, respectively. 
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Recent research in cross-disciplinary fields involving statistical mechanics and information 
sciences has shown that methods and concepts from physics can be useful in the development 
and analysis of efficient algorithms for computing probabilities or solving combinational prob- 
lems.^ One of the most prominent examples in such research is the development of survey 
propagation (SP). 2 ' This algorithm approximately evaluates the microscopic averages of dy- 
namical variables in a feasible amount of time for a fixed sample of disordered systems utilizing 
the concept of replica symmetry breaking (RSB), which was discovered in the study of spin 
glasses.® SP has been reported to give excellent results in studies on spin glass models® and in 
various combinatorial problems promoting further extension of the applicable range.® It 
has also been shown that SP reproduces the one-step RSB (1RSB) solution of replica theory® 
when applied to the mean-field-type spin glass models.^* This clarity of behavior is an impor- 
tant feature of the SP algorithm. However, the nature of solutions found by SP for general 
systems remains poorly understood, which may restrict the range of possible applications. 

The aim of the present Letter is to partially resolve this problem. More precisely, a family 
of approximation algorithms employing the Bethe free energy formalisnP^ is proposed for 
replicated systems. Analytically extending the proposed algorithms with respect to the number 
of replicas x = 1,2,... to x G R under the simplest replica symmetric ansatz turns out to 
provide a general expression of SP, in which x plays the role of a parameter specifying the 
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size of a subgroup of replicas in the conventional 1RSB schemed In this way, SP can only 
converge to a point offered by the analytical continuity of stationary points of the x-replicated 
Bethe free energy. 

As a basis for proposed algorithms, consider a joint distribution of N dimensional state 
variables S = (Si, S2, ■ ■ ■ , Sjv)> as given by 

M N 

p(5)=z- i n^wn^), a) 

fi=i 1=1 

where ^^(S^) and ipi(Si) are termed the clique and local evidences, which are dependent on a 
certain subset of multiple components (clique) and a single component Si, respectively, and 
Z = J2s Tlw=i ^iii^fj) T\b=i ^1(^1) is the partition function. For simplicity, only the Ising spin 
systems S = {+1, — 1}^ are considered here, but extension to other cases is straightforward. 
In such systems, evaluation of the marginal probabilities, 

P(S l ) = Y, P (S), (2) 
S\s t 

is in general computationally difficult, where X\Y denotes a subset of X from which Y is 
excluded. The development of computationally tractable algorithms achieving an accurate 
approximation of eq. © is therefore of great importance. 
The Kullback-Leibler divergence (KLD), as given by 

KL(Q|P) = ^Q(5)ln||||, (3) 
S 

where Q(S) is an arbitrary test distribution of S, offers a useful guideline for developing 
such an algorithm. As KL(Q|P) is always nonnegative and minimized to zero if and only if 
Q(S) = P(S), the minimization of KL(Q|P) with respect to Q(S) generally yields the correct 
distribution Q(S) = P(S). Direct application of this algorithm, however, is not particularly 
useful for computing eq. © because P(S) itself is in general not computationally tractable. 
Instead, applying this variational (minimization) principle of KLD or its approximation to a 
family of tractable test distributions systematically leads to a number of potentially effective 
approximation algorithms. 

The central idea of the present treatment is the application of this principle not to the 
original system but to a family of replicated systems, as follows: 

P x ({S a }) = flP(S a ), (4) 

a=l 

where {S a } is an abbreviation of the set of replicated systems {S 1 , S 2 , . . . , S x }. The abbre- 
viations {Sf} and {S 1 ^} are also used to represent {S^ 1 , Sf, . . . , Sf } and {SL S'l, . . . , <Sv}, 
respectively. Equation Q) yields the following equality regarding free energy: 

-lnZ = F x (Q)-~KL(Q\P x ), (5) 
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which holds for arbitrary x = 1, 2, . . . and the test distribution Q({S a }), where 

T x{ Q) = \ E Q({S«})ln ( Qi \ Sa a }) y (6) 

{S a } n a =i (n M =i m s j Ui=i ms?) ) 

As the true free energy — InZ is constant, this implies that the minimization of KL(Q\P X ) 
is equivalent to that of F X {Q)- This yields the correct distribution Q({S a }) = P x ({S a }) = 
Yla=i P{S a )- Since Ila=i P(S a ) 1S simply a product of duplications of eq. the minimiza- 
tion of eq. © for the original system and J~ X {Q) for the replicated system are equivalent. 
The introduction of the family of replicas therefore provides no benefits as long as the search 
for the optimal test distribution covers all feasible functional spaces. However, the replicated 
formalism can potentially provide better approximation accuracy than the original algorithm 
when the test distributions are limited to a tractable family or when the cost functions are 
somewhat approximated, as the tractable family or characteristics of the approximated func- 
tion depend on the number of replicas x. 

To show this, we approximate J- X {Q) by the Bethe free energy^^ of the a>replicated 
system, as follows: 



where C(fi) is the set of elements that directly relate to clique \i and Ci is the number of cliques 
to which element Si is directly related. The test distributions b^^S^}) and bi({Sf}) are termed 
beliefs which approximate the marginals S{ ( S ,a }\{S ,a } Px({S a }) an d S{S ,a }\{s a } Px({S a }), 
respectively, when JF^ ethe is extremized. Note that since both of these marginals are reduced 
from the identical distribution P x ({S a }), the reducibility condition 

E WW) = }), (8) 

{s;}\{s?} 

must hold when Si is an element of Su- 

If the variable dependence in eq. (^Q) is represented by a cycle-free graph, eq. (J7| under 
this constraint agrees exactly with ^(Q) for the test distribution Q({S a }) = Yif = b a '~ 1 ({s a }) ' 
in which case extremizing eq. Q leads to the exact assessment of eq. @ PES) Unfortunately, 
J- X {Q) and ^ etiie {\pa\, {bi}) do not accord in general. However, this indicates that the sta- 
tionary point of eq. (J7J) may provide a good approximation when the influence of the cycles 
can be regarded as weak. 

Extremizing eq. ((7J) with respect to the beliefs, adding the terms 
E { 5 f }V({^})(E { 5; } \{S ? }V({^})- b K{^})) ) where are the Lagrange 
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multipliers imposing constraint ijFfy. yields 

mU » ]) E { ^ } nLi^(^)n e ^)(^ ({s " }) n:=i^(5f))' u 

M rn H1 na=lV , K^f)II/,g^(O e ^ n ^ 



Then, inserting eqs. (jHJ and (jlt)j) into eq. (jSJ affords the stationary point condition of the 
Lagrange multipliers, which can be read as 

e -um) oc y: fiMs$ n (^ ii{sn) fi^)), (ii) 



a=l 



e-V«Sf}) oc [] e -Wtff}), (12) 

where .M(Z) denotes the sets of cliques that directly relate to clique elements I and A^({S'"}) = 

CFT Si/&M(Z) ^({'S'f }) ~ 

A remarkable property of these replicated systems is the invariance of stationary conditions 

Ijllj) and (|12jl under the permutation of replica indices a = 1, 2, . . . , x. This naturally leads to 

the assumption that the stationary point possesses the same symmetry. In the case of Ising 

spin systems S = {+1, —1}^, this can be represented as the simplest replica symmetric (RS) 

ansatz on the Lagrange multipliers, 



o=l 



e_Um) k / M*^ K \? E Z%L ' (14) 

7 (2 cosh(/i)J a: 

without a loss of generality,^ where 7r;_^(/i) and Tta-*i(h) are distributions that absorb the 
degree of freedom of the Lagrange multipliers. Inserting these multipliers into eqs. 1)11(1 and 
lfT2*|) provides a couple of saddle point equations in which Tc^i(h) and 7!7_^(/t) are solved 
explicitly for given 717^ (/i) and n^i(h), respectively. The natural iteration of the resultant 
equations yields a family of algorithms that are parameterized by the number of replicas 
x = l,2,..., 

^(Wn^*i-^) [ems,) n ^ ) j^-^^v})) .as) 

cx (2 cosher / n f^bfi ^-^- E U ( 16 ) 

J rkjinw (2 cosh \h v )) x ^777,\\ 
where t represents the number of iterations, M{^je£(»V }) = 
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'EsjSiMSi)' 



£5 Siipfi(S^) rije£( M )\i 


( e h 3 S 3 \ 


1 2cosh(hj) J 




e h 3 S 3 \ 


2cosh(hj) 1 



tanh" 1 J ^7 777 J and h? = tanh" 



There are two features to note here. First, the beliefs can be evaluated by inserting eq. ()13|) 
into eqs. © and (|TT7|) using the stationary solution of eqs. (fT5|) and Ijlfijl . Specifically, eq. (fTOjl 
can be assessed as h({Sf}) = J dhpi(h) ( e 2cos V(fe))* > yielding the following approximation of 
eq. ©: P(5,) ~ f dh Pl (h)^^, where 

, M cosh W r 1 iw fsg^ft - g - M 

Pl\fi) = — ; -j— . {*-•) 

I UpeM® { 2cosh(h^))^ 2 cos H h< l + E^M(l) K)7 
As the functional updates of eqs. (|15|) and ()16|) can be approximated by the Monte Carlo 
method on a practical time scale, these equations constitute a tractable algorithm for approx- 
imating the marginal probabilities (J2J). 

Second, although x = 1, 2, . . . has been assumed, the expressions of eqs. p5 j) -([17 |) can be 
analytically continued to x € R. In addition, the extremized values of the replicated Bethe 
free energy $^ethe _ Extr{{& j.^}} {^ ethe ({^ A t}, {&z})} can a l so De readily extended to x G R 
as follows: 

1 - 
--Yin 



jjjBethe 

/i=l 



1 - 

-^(l-Q)ln 



x 
1=1 



2 cosh(/i/) / 
2cosh(/i A1 ) 



(18) 

where 7r/_^(/i) and Tf^i(h) are the stationary solutions of eqs. Q15|) and Analytical 
extension of the variational functional (|7|. however, is nontrivial. Nevertheless, the analytically 
continued expressions of eqs. (|15|) and ()16|) eventually lead to a general expression of SP 
in which 7r*_^ (h) and 7r* are termed surveys, which is the main result of the present 

treatment. In this relation, x serves as the RSB parameter, which is introduced to represent 
the number of replicas in a subgroup in the conventional 1RSB formulation. For example, SP 
for the K-SAT problem of finite temperature, corresponding to eqs. (CI) and (C2) in ref. 1121 

_2 / 3TT x - J l S l 

can be derived from eqs. (fl6|) and (|15|) by setting x — ► m, ^7S7 = e ~ zpih ^^^ and 
ipiiSi) = 1 in conjunction with rescaling fields as 4 — > h and h — ► h, where [5 is the inverse 
temperature and J? £ {+1, —1} are randomly predetermined constants. The derivation of SP 
for the famous Sherrington-Kirkpatrick (SK) modeP^ is shown briefly in the appendix. 

It has been reported that extremizing eq. (|18|) with respect to the RSB parameter x 
provides a reasonable description of equilibrium states in the conventional replica theory.^ 
However, it may be difficult to deduce such a principle for the determination of x using 
only the framework presented here. The derivation is complicated by the dependence of the 
approximation accuracy on x, which is generally related to the specific characteristics of the 
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target system in a nontrivial manner, although certain extra constraints related to the concept 
of thermodynamic limits (e.g., homogeneity^) may be utilized for problems in the physics 
literature. 

The formalism presented here is related to other algorithms as follows. Equations (|15|) 
and (|16|) always have special solutions of the forms 7r* (/i) = 8(h — h*_^ ) and ir i(h) = 
8(h — ft*_^) 3 independently of x. The parameters h\^^ and are updated by ^tt^ = 

({^'eCO-OV-^' M->/u = ^? + Si/&M(J)\m Notice that these forms are an expression of 

the belief propag ation (BP^PESJ for eq. (QJ, the fixed point of which is usually linked to the 
variational condition of the conventional Bethe free energy ^" Bethe = ^BetheEl j n ^ ne current 
formalism, on the other hand, BP can be characterized as the solution of the highest symmetry 
obtained assuming the Lagrange multipliers of the limited form e~ PJ^ =1 ipi(Sf) oc 
% —^rir—^ , e~^ l ^ s i^ oc f ^ a_1 ' , which exist for arbitrary RSB parameter x G R. 
This relationship between BP and the assumption of eqs. (|13|) and (|14|) is analogous to that 
for the RS solution under the 1RSB ansatz in the conventional replica theory employed in 
spin glass research. 

The present algorithm can also be extended to the reduction of replica symmetry. For 
example, introducing the 1RSB ansatz^J on the Lagrange multipliers, we obtain 

x 

e- X ^llMS?) oc J Ptt^W J dh°*{h°) e {2coMha))m ) , (19) 

e -\i{{S?}) K / Pvrn^Jvrl TT / dh a Mh a )- s , (20) 

■/ Q =i V WM^))"^ 

where X(a) (a = 1,2, ...,m) denotes subsets of replica indices of an equal size m while 
Z>7!TIj_ > , t [7r] and T>Tttl^i[Tr] represent the variational measures of distributions vr(/i) and Tt(h), 
respectively. Analytically continuing this yields the conventional two-step RSB solution in the 
case of the SK model. This indicates that generalizing this provides an algorithm corresponding 
to the r + 1-step RSB solution of conventional replica analysis under the assumption that the 
Lagrange multipliers are of the form of the r-step RSB. 

In summary, a framework for the construction of a family of mean-field-type approxima- 
tion algorithms was derived by introducing the Bethe free energy formalism for x-replicated 
systems. Analytically continuing the algorithm obtained for x = 1,2,... to x G R under the 
replica symmetric ansatz leads to a general expression of survey propagation for such systems 
(eq. (fTjl). in which x plays the role of a replica symmetry breaking parameter in the 1RSB so- 
lution of conventional replica analysis. Belief propagation and generalized survey propagation 
can be reproduced from an identical variational functional (eq. (j7J)) corresponding to various 
levels of RSB ansatz on the replica symmetry of the Lagrange multipliers. This may be useful 
for clarifying the relationships between solutions obtained using these various algorithms. 
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Although the focus here was on algorithms derived from natural iteration of eqs. (jllj) and 
(|12|) . a range of schemes for the minimization of the Bethe free energy are available! 18 ^ 19 ^ 
Furthermore, it is known that the Bethe free energy formalism itself can be generalized to 
a more advanced scheme as the cluster variation method P Extending the present results to 
such schemes is a target of future work. 
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Appendix: SP for the SK Model 

The Sherrington-Kirkpatrick (SK) model is an Ising spin system characterized by the 
Hamiltonian H(S) = — Ez>fc JikSiSk — Ylb=i Hi^h where J/^ is sampled from an identical 
normal distribution Af(Jo/N, J 2 /N) independently of the unordered pair (Ik), and Hi repre- 
sents an external field. Identifying (Ik) with [i leads to ip^Sa) = e^ JlkS ' Sk and i^i(Si) = e@ Hl l . 

Two distinct properties of this system are the denseness of connectivity and the weakness 
of each coupling constant. Let 7r* ^(/t) be characterized by the two moments f dhrt^^^tyh = 
a\_+l and f dhTr t ^ l (h)(h — a^^) 2 = v* t . This representation allows eqs. (|TT)|) and (|T7j) to be 
expressed in terms of the tth update as 



„ Uh) . ^ (A „ 



fdh(2cosh(h)) x e 2A ^m 

and 

m a ^mri^ ( , 2) 

fdh(2cosh{h)) x e 2A t 

where = + E v6 X(I)V = E,/eA<(J)\AX-f $ = @ Hl + ^^M(i) a ^i 

and A* = YlfieMU) v ji-*l- ^ n turn, let us introduce f dhit\^^(h) tanh(fo) = rnj^^, 
f d/i7r*_^(/i)tanh 2 (/i) = Mf , f dhp\(h)t<mh.(h) = m\ and / dhp\(h) tanh 2 (/i) = Mf. Here, 
m\ is the estimated local magnetization at the ith update. The weakness of each cou- 
pling constant in conjunction with eq. lfT5)l indicates that a* j and u* t are updated as 
— /^M m fc-*/x> v l^ti — /^"^(^fc-^ ~ ( m fc^^) 2 )- Employing the law of large numbers, this 
implies that variances A* and Aj are updated independently of the pairs of indices (If/,) as 
A m „ N -ip2 j2 Y, ueMm ,(MU - (m}J J ) * N-ipJ* E^M(l)(M} - (m\) 2 ) ~ A* +1 ~ 
(3 2 J 2 (Q\ - Qi), where Q\ = N' 1 J2b=i M i and Qo = N ~ % E/Ii(™z) 2 - This also indicates 
that wz* and are related by ?n* ~ — /3 J M (1 — + 2?(Q* — Qo)) 771 !" 1 y i a ^ ne Taylor 
expansion. Inserting these results into eqs. (jA- 1|) and l)A-2j) leads to the following expression 
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of SP for the SK model: 



6 m 



k^l 



. f dh(2cosh(h)) x e 2f3 J (Q i- Q onanh(h) , x 

m i = — — (h > ( A4 ) 

fdh(2cosh(h))*e *P*HQ\-Ql) 

where r* = /3 2 J 2 (l - + x(Q$ - Q* Q )), and £ MeMW J ^ = £fc# J *fc is re P laced with its 
expectation (iV - l)iV _1 J 2 ~ J 2 using the law of large numbers. 

Notice that the fixed point condition of eqs. (|A-3j) and (|A-4j) is in agreement with the 1RSB 
analogue to the Thouless- Anderson-Palmer equation of the SK model3^ This demonstrates 
the consistency of SP with the 1RSB solution in conventional replica theory. 
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